Sparse Interpretible Audio Model

Table of Contents

• Architecture
• Sound Samples

Model Architecture

This small model attempts to decompose audio featuring acoustic instruments into the following components:

• A small (16-dimensional) global context vector
• Some maximum number of small (16-dimensional) event vectors, representing individual audio events
• Times at which each event occurs

Each sound sample below includes the following elements:

1. The original recording
2. The model's reconstruction
3. New audio using the original timing and context vector, but random event vectors
4. New audio using the original event and context vectors, but with random timings
5. New audio using the original timing and event vectors, but with a random global context vector

Future Directions

There are several areas that could provide further gains in compression and interpretibility:

• Imposing sparsity constraints on the number of events produced. You may notice that there are often many redundant events that could be merged into one.
• Performing vector quantization on the event vectors such that there is a discrete set of possible events

Cite this Work

                    
@misc{vinyard2023audio,
    author = {Vinyard, John},
    title = {Sparse Interpetable Audio},
    url = {https://JohnVinyard.github.io/machine-learning/2023/11/15/sparse-physical-model.html},
    year = 2024
}